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year. Teachers, in consultation with university faculty, increased the use of 
problem solving activities over 2 years using multiple strategies. Holistic 
and standardized assessment determined the overall effects of using action 
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a pretest and posttest, a whole-class selected problem test, and student 
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standardized measures in 1996 and 1 year later. Results indicated that during 
the second implementation year, most students successfully learned the 
selected strategies. However, when comparing them with matched controls on 
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Professional development schools have become part of the 
national effort to reform schools and improve the education of 
teachers (Darling-Hammond, 1994; Holmes Group, 1990; Vail, 

Cooper, & Frankels, 1997). Initially, universities embarked on school- 
university collaboration with their own needs in mind— a systematic 
strategy for handling placements for field experiences and faculty- 
designed research (Berg & Murphy, 1992; Brown, 1992; Zimpher, 
1990). Subsequently, restructured relationships emerged in school- 
university collaboration which were governed more by teacher and 
school interests (Lytle & Cochran-Smith, 1992; Zuelke & Nichols, 
1996). 

Today, hundreds of professional development schools operate to 
reform schools and improve the education of teachers through school- 
university partnering intended to be mutually beneficial. Experienced 
teachers need to continue to grow and interact with peers and other 
educators and to discover ways to enhance educational experiences for 
students. For universities, prospective teachers need excellent models 
in their preservice training to become good teachers and university 
faculty need teachers' perspective on school-based research. 

However, recent authors are calling for professional 
development schools to focus more on demonstrating that their 
activities are directly associated with improved student learning 
(Abdal-Haqq, 1998; Valli, Cooper, & Frankes, 1997). Simultaneously, 
draft standards for professional development schools (Levine, 1997) 
call for "continuous improvement supported by on-going practice 
based research" and "university and school-based teachers decid[ing] 
together what research focus they will... plan and 
implement... together." 

If there is a viable future for professional development schools, 
one means for generating commitment from policymakers and the 
public is to demonstrate that this school reform strategy yields 
significant improvement in student learning. The future of professional 
development schools is probably not found in a past that was perhaps 
satisfied with only new structures for student learning (e.g., whole 
language, interdisciplinary teaching). Educational policymakers and 
the public wants results that are associated with improved student 
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achievement. If teacher educators claim that action research in 
professional development schools improves student learning then 
evidence of this effect needs to be generated and widely disseminated. 
The need to focus professional development schools on student 
learning is an imperative of the present and the immediate future. 
Without the resolve to provide convincing evidence that action 
research in professional development schools improves student 
learning, it is unlikely that this school improvement effort will be long 
lived. 

Current literature on professional development schools has 
centered primarily on case descriptions of these developing models. 
Few studies have assessed the effects on K-12 students in professional 
development schools (Abdal-Haqq, 1998; Teitel, 1996; Vail, Cooper, & 
Frankels, 1997). Although the expectations are high that children in 
professional development schools achieve at higher levels of learning, 
only a few studies (Stallings, 1991; Wiseman & Cooner, 1996) have 
shown that improved student learning is an outcome of professional 
development school operation. Furthermore, there is scant evidence 
that this improved student learning would not have also occurred in 
matched control group classrooms and schools not associated with a 
professional development school. Clearly an urgent need exists to 
conduct more research to determine if students learn better in these 
schools. Without data which supports the belief that students achieve 
higher learning, it will be difficult for educational policy makers to 
advocate that professional development schools be widely replicated. 

At the same time that professional development schools are 
being created which include action-research on teacher practice, new 
national and state academic content and student performance 
standards are being developed and implemented. One set of these 
national content standards, the focus of this study, are those 
developed by the National Council of Teachers of Mathematics (NCTM, 
1989; 1991). These standards recommend that the study of 
mathematics emphasize problem solving so that students can better 
understand mathematical content. 

Thus fax, some evidence exists that classrooms which have 
responded to these mathematics' reforms have produced positive 
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student achievement results based on single or partial year 
instructional intervention (Carpenter, Fennema, Peterson, Chiang, & 
Loef, 1989; Cobb, et al., 1991; Hiebert & Weame, 1993). However, 
Wood and Sellers (1996) found that students needed to learn 
mathematics in reform-based classrooms for two years before 
significant differences in student achievement in mathematics could 
be observed. Moreover, in these studies, teachers transformed the 
entire mathematics program. It is not known what the results will be 
when teachers incrementally introduce these reforms--maintaining a 
textbook approach to teaching mathematics while simultaneously 
introducing problem solving strategies through a structured set of 
ancillary curriculum materials. Since this latter approach is likely to be 
an important step for the incremental introduction to new 
mathematics standards, it is important to understand if this approach 
provides significant changes in student learning. 

The creation of professional development schools which focus 
on national content standards offer teachers and university faculty a 
useful opportunity to conduct collaborative action research to 
determine if these standards are truly producing the intended 
changes in student learning. Cochran-Smith & Lytle (1993) note that a 
strength of action research is that teachers often develop or select the 
materials and assess the effects of their teaching on student progress, 
using multiple sources of data for analysis. Teachers who partner with 
university faculty in collaborative research have opportunities to field 
test new practices and reflect on the results of their work (Clark & 
Moss, 1996; Oja & Smuylan, 1989). Such data, including information 
on student achievement, can assist educators in determining the 
worth of adopted projects (Bickel & Hattrup, 1995). 

Furthermore, Slavin (1996; 1997) and Pogrow (1996) argue 
strongly that educators must assess the outcomes of school-based 
interventions in terms of student learning before recommending 
adoption by others. Similarly, Kimball, Swap, LaRosa, & Howick 
(1995), cited in Teitel (1995), caution professional development 
school representatives that 

the goal of improving students’ learning experiences 
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must take precedence over other aspects of partnership 
function, . . . [t]he means to effective partnership can easily 
become ends in themselves. For example, the energy for 
change in schools may become focused only on improving 
working condition for teachers, establishing more collabo- 
rative decision making structures, or creating more 
flexible schedules, all of which can be a means to an end of 
learning but cannot be ends in themselves. Administrative 
practice can change without passing advantage to the 
classroom, (p. 24). 

In Abdal-Haqq's (1998, p.31) view, "if children sure not 
significantly benefiting from the investment of time, effort, and 
resources devoted to PDSs, then both children and investors sure being 
betrayed... human and fiscal resources... are squandered if PDS 
implementers do not at least attempt to devise, test, refine sind 
document effective curriculum and practices." 

What are effects on student learning when a professional 
development school team introduces reforms in mathematics 
instruction? In this study, a professional development school team 
identified the content area of mathematics as an sirea of student need. 
Using the content standards of the NCTM as the guide for its work, 
the professional development school team selected instructional 
materials designed to support their students in problem solving while 
at the same time a textbook approach to instruction continued. 
Teachers, in consultation with university faculty, increased the use of 
problem solving activities with students over two years and measured 
the effects on student performance. The study, using a collaborative 
action-research model, reports the results of the second year of this 
effort. For one of these teachers, the results of students who were 
engaged in new mathematics instruction for two years is shared. The 
first year of this intervention is presented in an earlier paper (Devlin- 
Scherer, et al., 1997). 
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Method 

The Professional Development School Network 

In October of 1994, the School of Education and Professional 
Studies at Central Connecticut State University was funded by the 
Office of Educational Research and Improvement of the United States 
Department of Education to improve student achievement through 
school-based reform and to enhance the preparation of teachers who 
work in urban schools. The vehicle for enacting these primary goals 
was the creation of a set of professional development schools in the 
area adjacent to the university. Performance standards in the fields of 
mathematics and English /language arts provided the focus for school 
improvement projects. Professional teaching standards, developed by 
the National Board for Professional Teaching Standards and the 
National Council for the Accreditation for Teacher Education guided 
the effort to improve the preparation of teachers to teach in urban 
settings. 

Ten elementary, middle, and high schools from the Hartford, 
Connecticut region formed a network of professional development 
schools to launch student standards-based action research school 
improvement projects and simultaneously guided the improvement of 
the university's teacher education program. The results of one 
elementary school's second year (1996-1997) implementation of its 
mathematics reform action research project is the focus of this study. 
Two second grade teachers, three third grade teachers, one fourth 
grade teacher, and one fifth grade teacher systematically implemented 
an ancillary problem solving curriculum in their classrooms and 
completed data collection activities. 

Sample and Design 

The elementary school in this study is part of a school district 
with a 64% minority population. District elementary school 
enrollment includes 5000 children. At the district level 65% of the 
students live in homes with non-English as its primary language and 
56% receive free or reduced-priced meals. The sample K-5 
elementary school includes 480 students of which 54% represent 
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minority populations. Of that number 43% are Hispanic and 1 1% 
African-American. In the sample school 54% of the students reside in 
homes with non-English being spoken and 42% receive free or 
reduced-priced meals. In this setting, teachers volunteered to 
participate in professional development and classroom 
implementation activities. 

One dimension of the evaluation of the mathematics program 
intervention compared students at the sample school to matched 
students at similar schools in the district. The schools were nearly 
identical in the percentage of students receiving free or reduced- 
priced meals and closely aligned on the mathematics achievement on 
the most recently administered Connecticut Masteiy Test. Individual 
students were matched on the basis of standardized test scores 
routinely administered by the school district. Prior to the full year 
implementation, students in sample classrooms who had taken the 
Spring 1996 Stanford Achievement Test in mathematics were 
matched with students in control schools. 

Implementation of the ancillary mathematics program began in 
the Fall of 1996 and continued through June 1997. A matched pairs 
strategy was employed in order to allow for year-to-year control group 
comparisons and to provide a vehicle for longitudinal follow-up of as 
many students as possible who were identical in mathematics 
academic achievement before the program was introduced. 

Mathematics Program Intervention 

The problem solver activities for learning problem-solving 
strategies (Hoogeboom & Goodnow, 1987) was selected for the 
ancillary mathematics program intervention. Ten problem-solving 
strategies were taught and practiced by second-fifth grade students 
throughout the school year. The origin of providing systematic 
instruction and practice in problem solving in mathematics comes 
from the work of Polya (1973) and more recently has been promoted 
by the NCTM. The selection of these specific problem solving 
curriculum materials were based on positive comments from 
representatives in a nearby district which had used the program for 
several years. 
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The program has students follow a basic process for solving 
problems including understanding the problem, devising a plan, 
carrying out the plan, and looking back. Additionally, ten categories of 
problems which students encounter in the study mathematics are 
directly taught and practiced. Examples of these categories are make 
an organized list, table, pattern, or picture; guess and check; and 
logical reasoning. A total of 120 problems comprise these curriculum 
materials and permit teachers to develop various instructional 
implementation strategies during its full year of use. 

Teacher Development: Years 1 & 2 

Spring 1995 Mathematics Workshops . Before beginning 
classroom implementation activities in Year 1, one university project 
facilitator provided three workshops at the sample school. The 
contents of these workshops included the NCTM standards, using 
hands-on mathematical manipulatives for teaching of fractions and 
measurement, and strategies for teaching problem solving. 

Summer Institutes . Three of the sample teachers and a 
university faculty project facilitator, who had delivered the Spring 
1995 mathematics workshops at the sample school, completed a five 
day summer institute in 1995. At this institute the NCTM standards 
were shared and procedures for creating and carrying out an action 
research project were explained. At the conclusion of the summer 
institute the teachers and the university facilitator had developed a 
written plan for implementing and evaluating the use of new 
mathematics standards in classrooms focused on improving problem 
solving. 

Five additional site-based days during this summer were used by 
the institute team to further develop their implementation and 
evaluation plan. The ten problem solving strategies were discussed, 
the use of math manipulatives and cooperative learning was explored, 
a holistic scoring rubric was developed, and the concepts of using 
student interviews and mathematics journals were developed. The 
overall goal of these site-based days were to provide time for the 
teachers to make final preparations for the full-year of implementation 
and evaluation activities. 
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During the 1996 summer institute most of the seven Year 2 
teachers and two university facilitators worked within the summer 
institute setting and during site-based days to interact as a team with 
other network professional development schools and to plan the 
1996-1997 mathematics program implementation. 

Study Group Meetings and Related Professional Development. 
During the 1996-1997 school year the nine-person team met bi- 
weekly (the team met weekly during the 1995-1996 school year) to 
support full implementation of the action research plan and the data 
collection procedures. The team's meetings included sharing 
classroom successes and concerns, providing additional training, 
coordinating the logistics of assuring that teachers had appropriate 
copies of curriculum materials, implementing the testing and scoring 
procedures, and discussing the results of the Year 1 intervention. 

During this second year of implementation the team, as they had 
during Year 1 , visited another school in the region implementing 
problem solving instruction in mathematics and presented the Year 1 
intervention results at two national conferences. Team members 
attended professional development network forums which focused on 
sharing project strategies across school sites engaged in various action 
research interventions. Additionally, as they had during Year 1, the 
te am participated with schoolwide faculty in several professional 
development workshops related to implementing both ther textbook 
and problem solving components of the mathematics curriculum. 
Finally, the team sponsored a second annual family math night for 
project classrooms and produced a monthly newsletter on problem 
solving which was distributed to parents. 

Teacher Intervention Strategies 

Multiple and flexible strategies were employed to infuse problem 
solving in mathematics in the curriculum using the selected 
intervention materials. All seven teachers had the opportunity to adapt 
the problem solving curriculum materials to their own classroom 
setting. Teachers used a direct instructional strategy, with the use of 
an overhead projector, moving from directly teaching the strategy in a 
full group to students solving each problem independently. Some 
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teachers used cooperative learning and assigned some problems for 
homework. One third grade teacher describes her Year 2 intervention 
strategy as follows: 

I try to do the Problem Solver as a daily Math Journal exercise. 
Most weeks I will do from three to five examples. We do the first 
48 e xam ples as a whole class activity. These first 48 examples 
list the strategy it is teaching and I feel it is important to model 
the correct way to solve the problem and to explain how you got 
your answer. The examples from #49 to #120 do not tell the 
students what strategy to use and sometimes more than one 
strategy is implemented to solve the problem. 

I read the problem with the class while I work with a 
transparency. We circle the question and underline the data in 
each problem. I give the class time to solve the problems on 
their own from #49 to #120. I check the class answer by walking 
around while they are working out their answers and sometimes 
students with the correct answer will model it on the overhead 
transparency for the class. 

I have completed all 120 problems in the third grade Problem 
Solver in the last two years of doing the program. I am now [Year 
3, February 1998] half way through the book and it is half way 
through the school year. The Problem Solver takes anywhere 
from 15 minutes to 45 minutes daily. Starting as soon in 
September as possible is working out well. 

Assessment Instruments 

Holistic and standardized assessment instruments were used to 
determine the overall effects of using action research via a professional 
development school to improve student learning in mathematics. A 
pre- and post- test was administered to treatment classrooms in Fall 
1996 and Spring 1997. This test consisted of problems matched to 
the problem solving strategies embedded in the ancillary curriculum 
materials. Two additional routine problem closely aligned to the type 
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of problems found in the textbook-based curriculum was also included. 
A holistic scoring rubric, comprising a scale of one to four, was used to 
score each pre-post and selected problem test. 

After teaching a problem solving strategy, students were 
administered a whole class selected problem test to determine their 
level of learning (see Devlin-Scherer, et al., 1997 for Year 1 results). 
Fin ally , randomly selected high, medium, and low achievement 
students were interviewed at the beginning and end of the year and 
asked to mark a four-point Likert-scale instrument indicating their 
perceived competence in a problem solving approach to mathematics. 
The scale ranged from "I do not understand how to solve problems. I 
do not know where to begin. I am usually stuck,” to "I understand how 
to solve problems. I can clearly explain my answer. I always have 
acceptable answers” (see Devlin-Scherer, et al., 1997 for Year 1 
results). 

Two sets of standardized measures were utilized. The first was 
the Stanford Achievement Test (SAT)— sub-test Mathematical 
Applications. Both treatment and control students in grades three, 
four and five (test data was not available for grade two) were 
administered this test in May 1996 and again in May 1997. The 
second was the Connecticut Mastery Test (CMT)— Mathematics 
Section— administered to students in Connecticut beginning in grade 
three (no data was available for grade two students) in the September 
of each year. Students in the treatment and control grade classrooms 
completed this test in September 1996 and again in September 1997. 

Analyses 

The second year problem solving holistically scored data were 
analyzed using t-tests to compare pre-post changes for students in the 
classrooms of the seven sample teachers. 

For the SAT instrument, 1996 to 1997 raw score ANOVAs 
determined if treatment and control students had made significant 
gains in grades three, four, and five. The raw score results of the 1997 
CMT compared treatment and control students using ANOVA 
procedures. Finally, a third grade classroom's treatment and 
associated control students, who had begun the intervention in 1995, 
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were compared using 1995-1997 SAT data and the results of the 1997 
CMT. 

Results 

Holistic Measures 

Tables 1. 1-1.8 report comparisons of Fall to Spring holistic test 
scores for students. These yielded significant positive results for all six 
strategies as well as the two routine problem examples. Of the 55 
comparisons 42 produced significant results. Grade four and five 
students did not produce significant results on the routine problems 
and grade five students did not yield significant results for several of 
the non-routine problems. Overall, however, teachers taught these 
strategies well, and as measured by pre-post tests highly related to the 
actual problem solving strategies, students learned these at significant 
levels in a majority of classrooms. 

Insert Table 1. 1-1.8 about here 



Standardized Measures 

Stanford Achievement Test . For treatment and control group 
students completing the Stanford Achievement Test (Table 2) neither 
group produced significant results. 

Insert Table 2 about here 



Connecticut Mastery Test . The CMT analyses (Table 3) indicates 
a significant result favoring the grade four control students in 
measurement/geometry. There were no other significant differences 
between treatment and control students on the total test nor the four 
sub-tests analyzed. 



Insert Table 3 about here 



Long Term Effects for 1995 Third Grade Classroom. When 
comparing the 1995-1997 SAT results for these Year 1 students, 
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neither the treatment or control groups yielded significant changes. In 
the same fashion when analyzing the 1997 CMT data for these groups 
there appeared to be no significant difference between them on the 
total test and the sub-tests. 



Insert Table 4 about here 



Discussion 

During the second year of the implementation of this problem 
solving mathematics project evidence shows that a majority of 
students successfully learned the six selected strategies. However, 
when comparing the students with their matched controls on a 
nationally standardized measure and a state administered assessment 
test, there appeared to be no differences between the two groups. 
Testing the concept that it may take two years for differences to 
emerge between treatment and control classrooms was not confirmed 
with the students from a Year 1 third grade classroom. 

How can these results be explained? That students learned 
these six strategies of problem solving is a testament to the good 
teaching they received throughout the school year. Most of the 
teachers developed an appropriate methodology for implementing this 
ancillary problem solving curriculum. 

The disappointing results on the standardized measures perhaps 
can be explained in a variety of ways. The amount of time devoted to 
teaching and practicing the problem solving strategies, while at the 
same time maintaining high levels of implementation of the textbook- 
based curriculum may not have been adequate at least to attain positive 
results on the SAT and CMT measures. Perhaps the somewhat high 
SAT pretest scores created a ceiling effect that thwarted anticipated 
differences between treatment and control students. Or perhaps the 
matched control students made gains in their mathematics 
achievement because some of their teachers were participating in 
mathematics instructional improvement activities with an area 
consultant. 
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It is also important to note the difficulty in maintaining intact 
student samples in this urban district. Treatment students were 
matched with controls the Spring of the year before implementation 
began on the SAT test, proceeded through the implementation year 
and tested in the Spring again on the SAT, and then were 
administered the CMT the following Fall. The grade five teacher, for 
example, worked with 25 students, although only 19 took both SATs 
and only 10 were left in the district to compare their CMT results. 

Loss of sample was also a factor in the results of the comparisons made 
for the grade 3 students over a two-year period. Some teachers noted 
that the students lost from the samples were those who made the 
greatest gains. With sample sizes so low, differences between 
treatment and control groups needed to be much greater for these 
comparisons to be significant. 

There also appears to be little theoretical evidence that students 
who learn these strategies well will translate the results on more 
traditional standardized measures of mathematical achievement. 
Although the teachers in this study maintained their use of the 
textbook curriculum in a variety of ways, they did not emphasize the 
more routine problems encountered on standardized tests and this 
may have been associated with less impressive results on the 
standardized measures. If the NCTM Standards , particularly in 
problem solving, are to become widely accepted and translated into 
curriculum and instructional materials, it will be important for high 
stakes standardized measures of achievement to be in close 
congruence with these problem solving strategies. 

Finally, it may be essential for action research projects 
associated with professional development schools to select 
interventions which have demonstrated prior success in student 
learning on standardized measures before the intervention is selected 
for adoption. Much time and effort is devoted to planning, 
implementing, and evaluating an intervention described in this study. 

It would be prudent to not adopt curriculum materials only based on 
the positive comments of others. Instead, a careful review of the 
results of the program for significant positive effects for treatment 
compared to control classrooms would give teachers and university 
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faculty more research-based reasons to assume the intended 
intervention will likely cause the expected positive results for student 
learning. 

This study begins to address the need for professional 
development schools to demonstrate that the collaborative activity of 
schools and universities does produce significant positive changes in 
student learning. If action research on new instructional strategies is a 
vehicle for improvement of the quality of education in our nation, 
perhaps professional development schools can play a role in 
introducing these collaborative methods of inquiry within their 
network schools. Coupling implementation of new instructional 
strategies with a carefully designed action research model may be a 
powerful approach to ensure that students maximize their learning. 
More research will be needed to determine if this approach to school 
improvement merits further development and additional resources. 
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TABLE 1.1 

Comparison of Fall 1996 and Spring 1997 Problem Solving Holistic Test Scores 

for Grades 2-5 Treatment Classrooms 



Problem Solving 
Strategy by 
Grade 


n 


Mean 
FaU '96 
(SD) 


Mean 
Spring '97 
(SD) 


t 


P 


Organized List 












Grade 2 (A) 


24 


1.04 

(.20) 


2.84 

(.94) 


-9.33 


.000*** 


Grade 2 (B) 


23 


1.04 

(.20) 


2.29 

(.69) 


-8.51 


.000*** 


Grade 3 (A) 


18 


1.28 

(.46) 


2.89 

(.90) 


-6.76 


.000*** 


Grade 3 (B) 


24 


1.96 

(1.20) 


3.88 

(.45) 


-7.35 


.000*** 


Grade 3 (C) 


25 


2.67 

(1.31) 


3.52 

(.82) 


-2.72 


.010** 


Grade 4 


20 


2.15 

(.88) 


3.00 

(.86) 


-3.10 


.004** 


Grade 5 


25 1 











1 The Grade 5 teacher did not pre-post test for Organized List 
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TABLE 1.2 

Comparison of Fall 1996 and Spring 1997 Problem Solving Holistic Test Score: 

for Grades 2-5 Treatment Classrooms 



Problem Solving 
Strategy by 
Grade 


n 


Mean 
FaE '96 
(SD) 


Mean 
Spring '97 
(SD) 


t 


P 


Make a Table 












Grade 2 (A) 


24 


1.04 


3.17 


-10.57 


.000*** 






(.20) 


(.96) 






Grade 2 (B) 


23 


1.82 


3.52 


-6.12 


.000*** 






(1.01) 


(.81) 






Grade 3 (A) 


18 


1.33 


2.89 


-6.45 


.000*** 






(.69) 


(.76) 






Grade 3 (B) 


24 


1.08 


2.71 


-8.00 


.000*** 






(.28) 


(.96) 






Grade 3 (C) 


25 


1.13 


2.52 


-4.78 


.000*** 






(.34) 


(1.42) 






Grade 4 


20 


2.15 


3.25 


-3.77 


.001*** 






(.99) 


(.85) 






Grade 5 


25 


2.76 


2.76 


.00 


1.00 






(1.23) 


(1.20) 
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TABLE 1.3 

Comparison of Fall 1996 and Spring 1997 Problem Solving Holistic Test Score 

for Grades 2-5 Treatment Classrooms 



Problem Solving 
Strategy by 
Grade 


n 


Mean 
Fan '96 
(SD) 


Mean 
Spring '97 
(SD) 


t 


P 


Make a Pattern 












Grade 2 (A) 


24 


1.08 


2.83 


-10.56 


.000**’ 






(.28) 


(.76) 






Grade 2 (B) 


23 


1.22 


3.39 


-9.34 


.000*** 






(.74) 


(.84) 






Grade 3 (A) 


18 


1.83 


3.56 


-5.80 


.000**’ 






(1.04) 


(.71) 






Grade 3 (B) 


24 


1.33 


3.46 


-4.17 


.000** : 






(.64) 


(.62) 






Grade 3 (C) 


25 


1.50 


2.92 


-6.31 


.000**' 






(.51) 


(1.00) 






Grade 4 


20 


2.25 


3.00 


-2.38 


.023* 






(1.07) 


(.92) 






Grade 5 


25 


2.28 


2.88 


-1.94 


.059 






(1.10) 


(1.09) 
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TABLE 1.4 

Comparison of Fall 1996 and Spring 1997 Problem Solving Holistic Test Scores 

for Grades 2-5 Treatment Classrooms 



Problem Solving 
Strategy by 
Grade 


n 


Mean 
FaH '96 
(SD) 


Mean 
Spring '97 
fSD) 


t 


P 


Guess & Check 












Grade 2 (A) 


24 


1.87 


2.75 


-2.99 


.005*** 






(1.03) 


(.99) 






Grade 2 (B) 


23 


1.04 


2.79 


-8.99 


.000*** 






(.20) 


(.93) 






Grade 3 (A) 


18 


1.67 


3.06 


-4.37 


.000*** 






(.91) 


(1.00) 






Grade 3 (B) 


24 


2.25 


3.46 


-4.17 


.000*** 






(1.22) 


(.72) 






Grade 3 (C) 


25 


2.64 


3.24 


-1.78 


.082 






(1.19) 


(1.20) 






Grade 4 


20 


1.35 


3.55 


-8.85 


.000*** 






(.81) 


(.76) 






Grade 5 


25 


2.24 


2.84 


-1.64 


.110 






(1.23) 


(1.34) 








24 



24 



TABLE 1.5 

Comparison of Fall 1996 and Spring 1997 Problem Solving Holistic Test Score 

for Grades 2-5 Treatment Classrooms 



Problem Solving 
Strategy by 
Grade 


n 


Mean 
FaU '96 
(SD) 


Mean 
Spring '97 
(SD) 


t 


P 


Make a Picture 












Grade 2 (A) 


24 


1.42 


2.71 


-6.66 


.000**" 






(.58) 


(.75) 






Grade 2 (B) 


23 


1.04 


2.75 


-10.94 


.000**“ 






(.20) 


(.74) 






Grade 3 (A) 


18 


1.39 


2.44 


-4.27 


.000** = 






(.70) 


(.78) 






Grade 3 (B) 


24 


2.00 


3.46 


-6.26 


.000*** 






(.72) 


(.88) 






Grade 3 (C) 


25 


2.48 


3.36 


-4.07 


.000** = 






(.59) 


(.91) 






Grade 4 


20 


2.00 


2.65 


-2.16 


.038* 






(1.03) 


(.88) 






Grade 5 


25 


2.92 


2.80 


.44 


.660 






(.86) 


(1.04) 
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TABLE 1.6 

Comparison of Fall 1996 and Spring 1997 Problem Solving Holistic Test Scores 

for Grades 2-5 Treatment Classrooms 



Problem Solving 
Strategy by 
Grade 


n 


Mean 
Fan '96 
rsD) 


Mean 
Spring '97 
(SD) 


t 


P 


Logical Reasoning 










Grade 2 (A) 


24 


1.50 


3.13 


-7.14 


000*** 






(.66) 


(.90) 






Grade 2 (B) 


23 


1.04 


3.25 


-10.71 


ooo*** 






(.20) 


(.99) 






Grade 3 (A) 


18 


1.94 


3.56 


-4.74 


000*** 






(1.30) 


(.62) 






Grade 3 (B) 


24 


1.88 


3.58 


-6.84 


000*** 






(.99) 


(.72) 






Grade 3 (C) 


25 


2.44 


3.72 


-4.77 


.000*** 






(1.12) 


(.74) 






Grade 4 


20 


2.80 


3.30 


-1.86 


.071 






(.70) 


(.98) 






Grade 5 


25 


2.68 


3.28 


-2.16 


.037* 






(1.11) 


(-84) 
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TABLE 1.7 

Comparison of Fall 1996 and Spring 1997 Problem Solving Holistic Test Scorec 

for Grades 2-5 Treatment Classrooms 



Problem Solving n Mean Mean t p 

Strategy by Fall '96 Spring '97 

Grade [SDJ [SDJ 

Routine 1 



Grade 2 (A) 


24 


Grade 2 (B) 


23 


Grade 3 (A) 


18 


Grade 3 (B) 


24 


Grade 3 (C) 


25 


Grade 4 


20 


Grade 5 


25 



2.83 

(1.46) 


3.71 

(.69) 


1.04 

(.20) 


3.25 

(1.07) 


1.39 

(.96) 


2.44 

(.83) 


3.64 

(.57) 


3.92 

(.40) 


3.92 

(.28) 


3.88 

(.33) 


3.70 

(.57) 


3.60 

(.68) 


3.52 

(.87) 


3.80 

(.50) 



-2.65 


.012' 


-9.90 


.000 


-4.27 


.000 


-2.01 


.050' 


.46 


.650 


.50 


.620 


-1.39 


.170 
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TABLE 1.8 

Comparison of Fall 1996 and Spring 1997 Problem Solving Holistic Test Scores 

for Grades 2-5 Treatment Classrooms 



Problem Solving 
Strategy by 
Grade 


n 


Mean 
Fan '96 
(SD) 


Mean 
Spring '97 
(SD) 


t 


P 


Routine 2 












Grade 2 (A) 


24 


3.00 


3.71 


-2.17 


.037* 






(1.35) 


(.86) 






Grade 2 (B) 


23 


1.26 


3.65 


-10.23 


ooo*** 






(.86) 


(.71) 






Grade 3 (A) 


18 


1.89 


3.44 


-6.05 


ooo*** 






(.76) 


(.78) 






Grade 3 (B) 


24 


3.42 


3.96 


-3.11 


.005** 






(.83) 


(.20) 






Grade 3 (C) 


25 


2.96 


3.44 


-1.43 


.160 






(1.23) 


(1.12) 






Grade 4 


20 


2.70 


2.90 


-.55 


.590 






(1.34) 


(.91) 






Grade 5 


25 


2.84 


2.56 


.87 


.390 






(.99) 


(1.11) 
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TABLE 2 

Comparison of 1996 and 1997 Stanford Achievement Test Raw Scores 
(Mathematics Applications) for Treatment (T) and Matched Pairs Control (C) 

Grade Three-Five Classrooms* 



Grade 


n 


Mean 

1996 

(SD) 


Mean 

1997 

(SD) 


F 


P 


T Grade 3 (B) 


20 


26.35 

(5.72) 


29.90 

(7.43) 


2.87 


.098 


C Grade 3 (B) 


20 


26.35 

(5.72) 


27.90 

(5.65) 


.74 


.394 


T Grade 3 (C) 


16 


28.13 

(3.78) 


28.06 

(7.12) 


.00 


.975 


C Grade 3 (C) 


16 


28.13 

(3.78) 


30.19 

(4.74) 


1.86 


.183 


T Grade 4 


16 


26.94 

(6.67) 


26.94 

(6.67) 


.00 


1.000 


C Grade 4 


16 


26.94 

(6.67) 


28.63 

(7.78) 


.43 


.515 


T Grade 5 


19 


27.95 

(8.44) 


26.32 

(9.41) 


.32 


.577 


C Grade 5 


19 


27.95 

(8.44) 


27.53 

(7.92) 


.03 


.875 



* Grade 2 students' SAT data not available. Grade 3 (A) students were from a bilingual 
classroom; SAT data not available. 
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TABLE 3 

Comparison of 1997 Connecticut Mastery Test Raw Scores for Treatment 
and Matched Pairs Control Grade 3-5 Classrooms 



CMT Categories/ 
Grade 


n 


Mean 

Treatment 

(SD) 


Mean 

Control 

(SD) 


F 


P 


Concepts 












Grade 3 (B) 


17 


18.24 


18.17 


.00 


.955 






(3.56) 


(2.40) 






Grade 3 (C) 


14 


19.36 


18.14 


.86 


.361 






(1.91) 


(4.50) 






Grade 4 


14 


21.21 


22.71 


.41 


.527 






(5.96) 


(6.41) 






Grade 5 


10 


24.00 


22.70 


.11 


.745 






(10.07) 


(7.32) 






Number Facts/ 












Computation 












Grade 3 (B) 


17 


14.59 


14.71 


.03 


.861 






(2.09) 


(1.80) 






Grade 3 (C) 


14 


15.21 


14.86 


.41 


.526 






(1.25) 


(1.66) 






Grade 4 


14 


16.93 


19.14 


1.56 


.223 






(4.75) 


(4.64) 






Grade 5 


10 


22.20 


21.00 


.15 


.707 






(6.99) 


(7.06) 
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TABLE 3 CONTINUED 



CMT Categories/ 
Grade 


n 


Mean 

Treatment 

(SD) 


Mean 

Control 

(SD) 


F 


P 


Problem Solving 












Grade 3 (B) 


17 


53.59 


51.71 


.46 


.503 






(8.19) 


(8.01) 






Grade 3 (B) 


14 


55.36 


53.71 


.61 


.443 






(4.75) 


(6.29) 






Grade 4 


14 


41.57 


45.93 


1.02 


.322 






(13.23) 


(9.28) 






Grade 5 


10 


41.70 


40.90 


.02 


.887 






(12.74) 


(12.07) 






Measurement/ 












Geometry 












Grade 3 (B) 


17 


20.41 


20.59 


.03 


.867 






(3.71) 


(2.21) 






Grade 3 (C) 


14 


20.64 


20.64 


.00 


1.000 






(2.47) 


(3.61) 






Grade 4 


14 


11.85 


13.86 


5.08 


.033* 






(2.51) 


(2.18) 






Grade 5 


10 


15.00 


15.40 


.03 


.874 






(6.00) 


(5.08) 






Total 












Grade 3 (B) 


17 


106.82 


105.18 


.10 


.749 






(16.17) 


(13.46) 






Grade 3 (C) 


14 


110.57 


107.36 


.49 


.491 






(8.40) 


(15.02) 






Grade 4 


14 


91.57 


101.64 


1.39 


.248 






(24.11) 


(20.92) 






Grade 5 


10 


102.90 


100.00 


.04 


.843 






(34.89) 


(29.31) 
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TABLE 4 



Long Term Effects: Comparison of Stanford Achievement Test (SAT— 
Mathematics Applications) and Connecticut Mastery Test (CMT) Raw 
Scores for Treatment and Matched Pairs Control For Year 1 Grade 3 

Classroom Over Two Years 



Test 


n 


Mean 

1995 

(SD) 


Mean 

1997 

(SD) 


F 


P 


SAT 












Treatment 


16 


21.56 

(5.72) 


24.63 

(7.43) 


2.43 


.130 


Control 


16 


21.56 

(4.35) 


24.75 

(8.47) 


1.79 


.191 



Test/ 

Category 


n 


Mean 

Treatment 

(SD) 


Mean 

Control 

(SD) 


F 


P 


CMT 1997 


Concepts 


13 


19.69 

(4.70) 


22.31 

(3.95) 


2.36 


.137 


Number Facts/ 


Computation 


13 


16.31 

(4.17) 


18.00 

(3.85) 


1.16 


.293 


Problem Solving 


13 


39.62 

(10.60) 


45.39 

(8.80) 


2.28 


.144 


Measurement/ 


Geometry 


13 


10.85 

(3.44) 


12.54 

(2.18) 


2.25 


.14/ 


Total 


13 


86.46 

(18.95) 


98.23 

(15.96) 


2.93 


.100 
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